E cient Computation of the Singular Value Decomposition with Applications to Least Squares Problems

We present a new algorithm for computing the singular value decomposition (SVD) of a matrix. The algorithm is based on using divide-and-conquer to compute the SVD of a bidiagonal matrix. Compared to the previous algorithm (based on QR-iteration) the new algorithm is at least 9 times faster on bidiagonal matrices of dimension n = 400, when running on a DEC Alpha with optimized BLAS. The speedup increases with dimension n. For the dense singular value decomposition, the speedup ranges from 2.2 to 3.9 for n = 400. When used to solve dense, square linear least squares problems, the operation count drops from 12n to 8 3 n, and the speedup ranges from 2.3 to 3.8 for n = 400. This means using the SVD for the least squares problem averages only 4.8 times slower than using simple QR decomposition, whereas it used to be over 15 times slower. We show how to modify the old least squares solver based on the SVD with QR-iteration to attain slightly better speedup, at the cost of O(n) storage. This makes the SVD a much more economical tool than it was before. Department of Mathematics and Lawrence Berkeley Laboratory, University of California, Berkeley, CA 94720. The author was supported in part by the Applied Mathematical Sciences Subprogram of the O ce of Energy Research, U.S. Department of Energy under Contract DE-AC03-76SF00098. Computer Science Division and Mathematics Department, University of California, Berkeley, CA 94720. The author was supported in part by NSF grant ASC-9005933, ARPA contact DAAL0391-C-0047 via a subcontract from the University of Tennessee, ARPA grant DM28E04120 via a subcontract from Argonne National Laboratory, and DOE grant DE-FG03-94ER25206. Computer Science Division, University of California, Berkeley, CA 94720. The author was supported by ARPA contact DAAL03-91-C-0047 via a subcontract from the University of Tennessee, and DOE grant DE-FG03-94ER25206.